Effects of payoff functions and preference distributions in an adaptive population 
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Adaptive populations such as those in financial markets and distributed control can be modeled 
by the Minority Game. We consider how their dynamics depends on the agents' initial preferences 
of strategies, when the agents use linear or quadratic payoff functions to evaluate their strategies. 
We find that the fluctuations of the population making certain decisions (the volatility) depends on 
the diversity of the distribution of the initial preferences of strategies. When the diversity decreases, 
more agents tend to adapt their strategies together. In systems with linear payoffs, this results in 
dynamical transitions from vanishing volatility to a non-vanishing one. For low signal dimensions, 
the dynamical transitions for the different signals do not take place at the same critical diversity. 
Rather, a cascade of dynamical transitions takes place when the diversity is reduced. In contrast, 
no phase transitions are found in systems with the quadratic payoffs. Instead, a basin boundary of 
attraction separates two groups of samples in the space of the agents' decisions. Initial states inside 
this boundary converge to small volatility, while those outside diverge to a large one. Furthermore, 
when the preference distribution becomes more polarized, the dynamics becomes more erratic. All 
the above results are supported by good agreement between simulations and theory. 
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I. INTRODUCTION 

Many natural and artificial systems consist of a pop- 
ulation of agents with coupled dynamics. Through their 
mutual adaptation, they are able to exhibit interesting 
collective behavior. Although the individuals are com- 
peting to maximize their own payoffs, the system is able 
to self-organize itself to globally efficient states. Exam- 
ples can be found in economic markets and communica- 
tion networks [E 0, S, H • 

As a prototype of an adaptive population, the Minority 
Game (MG) considers the dynamics of buyers and sellers 
in a model of the financial market in which the minority 
group is the winning one @. The agents adapt to each 
other through adjusting the payoffs of their strategies, 
wherein the payoffs summarize the collective behavior of 
the population when the environment changes. Theo- 
retical studies using the replica method [H, |a] and the 
generating functional @, H, Q successfully describe the 
statistical properties of these systems. However, since 
adaptation is a dynamical process, much of the attractor 
behavior remains to be explored. 

An important factor affecting the behavior of an adap- 
tive population is the dependence of the payoffs on the 
environment experienced by the individual agents. The 
payoffs help the agents to assess the preferences of their 
decisions, hence inducing them to take certain actions 
when they experience a similar dynamical environment 
in the future. Thus, the payoff function is crucial to 
the mechanism of adaptation. For example, the ways 
the agents evaluate their strategies in financial markets 
have a large influence on the market behavior. There 
are agents who focus their attention mainly on opportu- 
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nities of large profits, while others pay equal attention 
to all profitable opportunities large and small. Markets 
with individualistic agents may have diversified opinions 
about what the best strategies are. 

As an illustration of how payoffs influence market be- 
haviors, a payoff scheme based on expected price trends 
results in markets with a mixture of trend-followers and 
contrarians [lfj. The $-Game considers payoffs reward- 
ing correct expectations one step forward, giving rise to 
a self-sustained speculative phase [Tfj]. Bubbles, crashes 
and intermittent behaviors are also found in a similar 
extension of the MG [l^ ]. A recent extension of the 
MG considers agents rewarding trend-following strate- 
gies when the winning margin is small, and rewarding 
contrarian ones otherwise [l3|, Qjj]. As a result, non- 
Gaussian return distributions, sustained trends and bub- 
bles are found, reminiscent of real markets. The Wealth 
Game uses a wealth-based payoff scheme, and produces 
behaviors resembling those of arbitrageurs and trendset- 
ters, and markets with positive sums The Wealth 
Game payoff scheme enables the agents to have a strong 
history dependence when applied to Hang Seng index 
data. 

The agents in the original version of MG use a step 
payoff function 0, Oil E3] , meaning that the payoffs re- 
ceived by the winning group are the same, irrespective 
of the winning margin (the difference between the ma- 
jority and the minority groups). Latter versions of MG 
use a linear payoff function [5|, ||| 0, HI, Q , i n which the 
payoffs increase with the winning margin. Other payoff 
functions yield the same macroscopic behaviors in their 
dependence of the population variance on the complexity 
of strategies [H, [T^]. Thus, it appears that the behav- 
ior of the population is universal as long as the payoff 
function favors the minority group. 

However, when one considers details beyond the popu- 
lation variance, one can find that the agents self-organize 
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in different ways for different payoff functions. For a pay- 
off function that favors a large winning margin, the distri- 
bution of the buyer population is double-peaked . This 
shows that the dynamics of the population self-organizes 
to favor large winning margins of cither buyers or sellers, 
because the agents have adapted themselves to maximize 
their payoffs. 

Another important factor is the initial preference of an 
agent towards the individual strategies she holds. Con- 
sidering the example of financial markets, the agents may 
enter the market with their own preferences of strate- 
gies according to their individual objectives, expectations 
and available capital, even in the case that they hold 
the same set of strategies. For example, some agents 
have stronger inclinations towards aggressive strategies, 
and others more conservative. Hence, it is interesting to 
consider the effects of a diverse distribution of strategy 
preferences on the system behavior. Our recent work on 
MG revealed that when the diversity of strategy prefer- 
ences increases, the system dynamics generally converges 
slowly, but the maladaptation of the agents, as gener- 
ally reflected in the fluctuations of their decisions, can 
be greatly reduced [2(| Hl| . A scaling relation between 
the population variance and the diversity was found, but 
there are no dynamical transitions [20 . |21| . 

Besides the diversity of preferences, the profile of the 
diversity distributions also influences the dynamics of the 
system. For example, the Gaussian distribution of pref- 
erences studied in [2(| HH is a prototype of a continuous 
distribution, modeling a population with less polarized 
opinion. This distribution is different from the bimodal 
distribution studied in 0, H, HH, [2^, [24[ , which is more 
appropriate to model a population with polarized opin- 
ion. Both cases share many common statistical features, 
such as the reduction of fluctuations on increasing diver- 
sity. However, since adaptation is a dynamical process, 
one would expect that the bimodal case may have a much 
more erratic temporal behavior than the Gaussian case. 
This constitutes one of the purposes of our study. 

In this paper, we consider the attractor behavior of 
an adaptive population using linear and quadratic pay- 
off functions, with the distribution of initial preferences 
being either Gaussian or bimodal. While the statistical 
behaviors in these cases arc quite similar, their attractor 
behaviors are different, revealing the different dynamics 
by which a population adapts to its specific environment. 
A number of findings in this paper illustrate this point. 
For example, when the step payoff function is replaced 
by the linear payoff function, the scaling relation between 
population variance and diversity is replaced by a dynam- 
ical transition between vanishing and non-vanishing step 
sizes. For low signal dimensions, the dynamical tran- 
sitions take place in the form of cascades for different 
signal dimensions. Even when the cascades are blurred 
at higher signal dimensions, it is possible that there is 
a crossover from anisotropic to isotropic motion in the 
phase space. When the Gaussian preference distribution 
is replaced by the bimodal distribution, the dynamics 



becomes more bursty, and the phase space motion be- 
comes more jumpy. Going from linear to quadratic pay- 
offs, we find that the basins of attractor for vanishing and 
non- vanishing step sizes coexist. All these rich behaviors 
demonstrate the flexibility of an adaptive population for 
self-organizing to states in which agents maximize their 
payoffs and is, hence, important in modeling of economics 
and distributed control. 

The paper is organized as follows. After formulating 
MG in Section II, we consider the cases of linear and 
quadratic payoffs in Sections III and IV respectively, fol- 
lowed by a conclusion in section V. Detailed derivatives 
are presented in Appendices A to C. 



II. MINORITY GAME 

The Minority Game model consists a population of N 
agents competing for limited resources, N being odd 0]. 
Each agent makes a decision I or at each time step, 
and the minority group wins. For economic markets, the 
decisions I and correspond to buying and selling re- 
spectively, so that the buyers can win by belonging to 
the minority group, which pushes the price down, and 
vice versa. For typical control tasks, such as the distri- 
bution of shared resources, the decisions 1 and may 
represent two alternative resources so that fewer agents 
utilizing a resource implies more abundance. The de- 
cisions of each agent are responses to the environment 
of the game, described by signal n*{t) at time t, where 
fi*(t) = 0, ...,D — I. These responses are prescribed by 
strategies, which are binary functions mapping the D sig- 
nals to decisions 1 or 0. In this paper, we consider both 
endogenous signals and exogenous signals. In the endoge- 
nous case, the signals are the history of the winning bits 
in the most recent m steps. Thus, the strategies have 
an input dimension of D = 2"\ In the exogenous case, 
the signals are randomly generated from the D possible 
choices at each time step. The parameter a = D/N is 
referred to as the complexity. 

Before the game starts, each agent randomly picks s 
strategies. Out of her s strategies, each agent makes de- 
cisions according to the most successful one at each step. 
The success of a strategy is measured by its cumulative 
payoff, as explained below. 

Let = ±1 when the decisions of strategy a are 1 
or 0, responding to signal /i. Let a*(i,t) be the strategy 
adopted by agent i at time t. Then, 

i 

is the excess demand of the game at time t. The payoff 
received by strategy a is then -£f {t) (p{VNA{t)), where 
ip is the payoff function. The step and linear payoffs are 
tp(x) = sgnx and x, respectively. Let ft a (t) be the cu- 
mulative payoff of strategy a at time t. We will consider 
both online and batch updates of the payoffs. The online 
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updating dynamics is described by 

n a (t + 1) = n a (t) - ^'v^VNAit)), 

whereas the batch updating dynamics is given by 

n a (t + 1) = n„(t) - ]T gipiVNA*®), 

wherein A^{t) is a /^-dimensional vector 



(1) 



(2) 
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The diversity of initial preferences of strategies is in- 
troduced by initializing the cumulative payoffs f2«j(t) of 
strategy a (a— 2, s) of agent z at time t = to ran- 
dom biases Wi a , with fii a (0) = for a = 1 [13,[2l|. Thus, 
the preferences influence the score of each strategy for 
the rest of the game. In this paper, we will consider 
both the Gaussian and delta distributions of the initial 
preferences of the strategies. In the Gaussian case, the 
preference distribution has a mean and variance R, 



PM = 



ia/ 2R 



yph^R 



(3) 



The ratio p = R/N is referred to as the diversity. In 
the delta function case, uj ia = y/pN, as considered in Q. 
Since the system behavior is invariant with respect to 
random permutations of the strategies, the introduction 
of the delta function is equivalent to a bimodal preference 
distribution 



P(w fa ) = S(uia - yJpN)/2 + 5(u ta + y/ P N)/2, (4) 

with mean and standard deviation y/pN. As we shall 
see, the two preferences distributions have different ef- 
fects on the dynamics of the game. 

To monitor the mutual adaptive behavior of the popu- 
lation, we measure the variance a 2 /N of the population 
making decision 1, as defined by 



(A"*W(i))] s 



(5) 



where the average is taken over time when the system 
reaches steady state and over the random distribution of 
strategies and biases. 



III. MG WITH LINEAR PAYOFFS 

A. THE GAUSSIAN DISTRIBUTION OF 
STRATEGIES' INITIAL PREFERENCES 

1. The onset of instability. 

We first consider the case of online dynamics with a 
Gaussian distribution of initial preferences. As Fig. [1] 
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FIG. 1: a 2 /N versus a with linear payoffs for p = 0, 0.25, 1, 
and 4, for N = 251, s = 2, and 1000 samples. Inset: The 
corresponding plot for step payoffs. 
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FIG. 2: a 2 /N versus p with linear payoffs of a = 0.002, 0.01, 
and 0.02 for different m and N. Inset: a 2 /N versus p with 
step payoffs m = 1, and N = 127,511,2047, and 8191. The 
dashed-dotted line is the scaling prediction. For both payoffs, 
s = 2 and the number of samples is 1000. 



shows, the dependence of the variance a 2 /N on the com- 
plexity a for linear payoffs is very similar to that for step 
payoffs 0, [2l|. For a above a universal critical value 
a c (w 0.3), the variance drops when a is reduced. The 
effect of introducing the diversity is also similar to that 
for step payoffs; namely, the variance remains unaffected 
when a > a c , but decreases significantly with the diver- 
sity when a < a c . 

However, there are differences when one goes beyond 
this general trend. As Fig. [5] shows, the variance curves 
at different values of a cross at p = p c w 0.16, indicating 
the existence of a continuous phase transition at p c from 
a phase of vanishing variance at large p to a phase of 
finite variance at small p. This behavior is very different 
from that for step payoffs, where the variance scales as 
and there are no dynamical transitions (Fig.[5]inset). 

The picture is confirmed by analyzing the dynamics of 
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much less than y/R, 
A»(t+l) = A> t {t)- 



fNA»{t))5^ (t) . (6) 



For step payoffs, Eq. © converges to an attractor con- 
fined in a D-dimensional hypercube of size y/2/irR, irre- 
spective of the value of R. On the other hand, for linear 
payoffs, A^(t + 1) becomes a linear function of A^{t) with 
a slope of 1 — \j2pnp. Hence, for p > p c = 1/2tt ~ 0.16, 
the step sizes 

AA>*(t) =\ A»(t+1) - A»(t) | 

converge to zero. In particular for 1/2tt < p < 2/tt, the 
motion in the phase space converges with oscillations, 
whereas for p > 2/tt, the motion converges without oscil- 
lations. On the other hand, for p < p c , steps of vanishing 
sizes become unstable, resulting in a continuous dynam- 
ical transition at p c . 

The phase transition at p c illustrates how the agents 
adapt to the changing environment when the diversity 
changes. In general, at high diversity, only a small frac- 
tion of agents switches their strategies at each time step. 
This gentle movement results in, at most, small win- 
nin g m argins, as revealed in the example of step payoffs 
[20l |2l|. In contrast, the winning margins at low diver- 
sity can be larger. Since we are considering payoffs that 
are linear functions of the winning margin, the agents 
are adapted to pay more attention to decisions that re- 
sult in larger profits, leading to stronger responses at low 
diversity and vanishing responses at high diversity. 



FIG. 3: Attractor dynamics for m = 1 in (a) endogenous 
dynamics, and (b) exogenous dynamics with online update. 
p = 0.01 



the game for small m. The dynamics can be conveniently 
described by introducing the D-dimcnsional vector A^ (t) 
which is defined in Eq. While only one of the D 

signals corresponds to the historical signal p*(t) of the 
game, the augmentation to D components is necessary 
to describe the attractor structure of the game dynam- 
ics. Fig. El^a) illustrates the attractor structure in this 
phase space for the visualizable case of m = 1 with online 
update and endogenous signals. The dynamics proceeds 
in the direction that tends to reduce the magnitude of 
the components of A^(t) However, the components 
of A^{t) overshoot, resulting in periodic attractors of pe- 
riod 2D. For m = 1, the attractor is described by the se- 
quence p* (t) = 0,1,1, 0, and takes the L-shape as shown 
in Fig. [3]Ja) [2l| . Note that the displacements in the two 
directions may not have the same amplitude. This is also 
true for online update and exogenous signals as Fig. [3jb) 
shows, although it does not have periodic attractors like 
those in Fig. [3][a). 

Following steps similar to those in [2l|, we find that 
for m not too large and for convergence within time steps 



2. The time scales of convergence. 

The onset of the instability when p decreases is accom- 
panied by the separation of two convergence times. The 
first time scale is the state convergence time, which is 
defined as the number of time steps needed to reach the 
attractor (for m = 1, this is the sequence 0,1,1,0) 

The second time scale is the population convergence 
time which is defined for online update and endogenous 
dynamics as follows. Note that the excess demand A(t) 
may not arrive at the steady-state time series even af- 
ter the attractor states — sgnA(t) become steady. To 
measure the convergence time scale for A(t), the peri- 
odic trend of A{t) in the attractor must be subtracted. 
This is done by measuring the sample average of Ai — 
{A(t)) mod ( t ,2D)=i for i = 0, ...,2D- 1 and monitoring the 
time dependence of A(t) — ^4 m od(t,2D)- As shown in the 
inset of Fig. [U this difference converges exponentially to 
a baseline, and the inverse of the slope of the exponen- 
tial convergence yields the population convergence time. 
As shown in Fig. [H the state convergence time increases 
smoothly with diversity. On the other hand, the popula- 
tion convergence time is distinct from the state conver- 
gence time in the low diversity region. The turning point 
is between p = 0.13 and 0.25, indicating a relation with 
the dynamical transition of step sizes at p c . 
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FIG. 4: The dependence of the state and population conver- 
gence time on the diversity for online update, endogenous dy- 
namics, linear payoffs and Gaussian preference distribution. 
Inset: the relation between A — (A) and t from the transient 
to the steady state for p — 7.943. Parameters: m = 1, s = 2, 
and N = 255, averaged over 1000 samples. 



3. Cascades of transitions. 

However, when p < p c , the step sizes for each of the 
D signals may not be equal. To see this, we monitor the 
variance for each of the D signals and rank them. The 
rth maximum variance is then given by 



S r = large 



(j[{(A»)% = , Ht) ~ «^>U =M >( t) ) 2 ],r) ,(7) 



where large A( (/(/i), r) is the rth largest function f(p) for 
p = 0, D—l, As Figs.[5]|6]show, the step sizes for the D 
signals do not bifurcate simultaneously at p = p c , Rather, 
only the first maximum bifurcates from zero when p falls 
below p c , while the step sizes for the remaining D — 1 sig- 
nals remain small. When the diversity further decreases 
to around 0.05, the second maximum becomes unstable 
as well, and a further bifurcation takes place. For m > 2, 
there are further bifurcations of the third or higher order 
maxima, resulting in a cascade of dynamical transitions 
when the diversity decreases. 

This cascade of transitions is confirmed by analysis. 
For m = 1, we can generalize Eq. ^ to convergence 
times of the order \fR. Assuming, without loss of gener- 
ality, that A 1 bifurcates while A remains small. We find 
that the variance of the buyer population, as derived in 
Appendix A, is 



N 32 v ' ' 



V, A^ 1 = erf 



^AA 1 



(8) 



where A^ 1 is the step size responding to signal 1. As 
the inset in Fig. \5\ shows, the analytical and the simu- 
lated results agree down to p ~ 0.05. However, when the 
diversity decreases further, this simple analysis implies 
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FIG. 5: S r versus p for online update, endogenous dynamics, 
linear payoffs and Gaussian preference distribution. Inset: 
a 2 /N versus p. The symbols are the simulation, the dotted 
line the theory with one bifurcation, and the solid lines the 
theory with two bifurcations. iV = 1001, m = 1, s = 2, and 
1000 samples. 



that the variance will saturate to a constant N/32, but 
simulated results are clearly higher. This discrepancy is 
due to a further bifurcation of the minimum step size. 
This can be analyzed by considering the effect of a per- 
turbation 8 A {t) in the direction of ^4°. As derived in 
Appendix B, the accumulated perturbation becomes 



SA°(t + A) 



1 - 



= (1 



(AA 1 ) 2 
8p 



5A°(t). (9) 



At p = 0.0459, where A^ 1 = 0.9775, the coefficient on 
the right-hand side of Eq. ^ reaches the value 1, and 
SA°(t) diver ges on further reduction of p. Numerical it- 
erations of the analytical equations for A 11 if), averaged 
over samples of different initial conditions, yield the the- 
oretical curves in Fig. [5] and the inset, agreeing very well 
with the simulated results. Similarly, the agreement be- 
tween analytical and simulated results are satisfactory 
for m — 2, as shown in Fig. [H^a). 

For m — 3, as shown in Fig. [H^b), the bifurcation of 
Si, remains distinctive from those of other directions, 
and the picture of at least one cascade is valid. Fur- 
thermore, we observe that in the low diversity limit for 
m = 1, 2, 3, the maximum step size approaches the same 
value of N/16 following the arguments in Appendix A. 
We also want to mention that results show that although 
the attractor structure of online update with exogenous 
signals is different from that with endogenous signals, the 
dependence of S r and a 1 /N on p are totally the same for 
both signals. 

This shows that the cascades are a general feature of 
the adaptive dynamics of the agents. When the diversity 
decreases, the agents find it more profitable to induce 
large winning margins responding to certain, but not all 
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FIG. 6: St- versus p for online update, endogenous dynamics, 
linear payoffs and Gaussian preference distribution(a) m — 2, 
(b) m = 3. In both cases, N = 1001, s — 2, and 1000 samples. 



signals. At sufficiently low diversity, large winning mar- 
gins become possible responses to all, signals, resulting 
in the cascades of transitions. However, for large signal 
dimensions at higher m, the interference of the responses 
to different signals will blur the transitions. 



4- Batch update 

We close this section briefly mentioning the results of 
batch update. We found that the variance of the buyer 
population with batch update is larger than that with 
online update (those of endogenous and exogenous sig- 
nals update are the same) and the difference increase as 
a increases, as shown in Fig. [JJ We also found that the 
dynamical transition to non-vanishing step sizes in the 
case of online update was replaced by a gradual crossover 
around p ~ 0.2, and the cascades of transitions in differ- 
ent directions are replaced by a simultaneous crossover 
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FIG. 7: a 2 /N versus p for linear payoffs and Gaussian pref- 
erence distribution, (a) m = 1; (b) m = 4. In both cases, 
N = 1001, s = 2, and 1000 samples. 



around p ~ 0.2. 



B. THE BIMODAL DISTRIBUTION OF 
STRATEGIES' PREFERENCES 

In this section we consider how the dynamics depends 
on different distributions of agents' initial preferences 
of strategies. In the previous section we have studied 
the case of Gaussian distributions of preferences, which 
model populations with less polarized opinions. In this 
section, we replace the Gaussian distribution with a bi- 
modal distribution described by Eq. Q, which is more 
appropriate to model a population with polarized opin- 
ions. As we shall see, the latter shares many common 
statistical features with the former, such as the cascades 
of dynamical transitions (subsection 1) and the change in 
the isotropy of motion (subsection 2) when the diversity 
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changes. On the other hand, the latter exhibits more er- 
ratic temporal behavior, such as the changing fraction of 
time spent on locations of high and low volatility in the 
attractor (subsection 3) and the bursty dynamics (sub- 
section 4) . The theoretical dynamical equations for both 
batch and online updates are derived in Appendix C. 

1. Cascades of dynamical transitions 

We found that for batch updates, Fig. Ufa) shows that 
the cascade of dynamical transitions is present for small 
signal dimensions but disappears as signal dimensions in- 
crease. For large signal dimensions, step sizes for different 
signal dimensions bifurcate simultaneously at p ~ 0.09. 
As shown in Fig. [8fb), there is a large jump of S\, indi- 
cating that it is a discontinuous transition, while for the 
Gaussian case, it is a continuous transition. This discon- 
tinuous transition was found previously in Q, and the 
transition point is around p c ~ 0.06 with the magnitude 
of the jump scaling as a~ 5 . Results are similar for online 
update. 

2. Isotropy of motion 

For batch update, we have devised new global parame- 
ters to describe whether the motion in the phase space is 
isotropic at each update. For D = 2, the two parameters 
are 

U x = (([(AA 1 ) 2 - (A^°) 2 ] 2 ) t ) samplo 

and 

Vl = ((pA^A^TMsamplc, 

they measure the displacements along the axial or diag- 
onal directions in the phase space of A 1 and A Q . 

We can see from Fig. |5] that as diversity decreases, the 
axial parameter grows from nearly zero to nonzero val- 
ues at around 0.2, while the diagonal parameter remains 
small. This shows that at each time step, the system 
responds to only one signal. When p decreases to about 
0.06, the diagonal parameter becomes nonzero too, show- 
ing that the system responds to more than one signal at 
each time step, and the motion becomes more isotropic. 
This cascade behavior is also well supported by our the- 
ory displayed on the same figure. 

For higher signal dimensions, the isotropy of motions 
in the attractors can be described by ranking the D com- 
ponents of randomly picked step sizes. The ranked com- 
ponents of different samples, as shown in Fig. 1101 exhibit 
several classes of behavior with increasing isotropy. Sam- 
ples in Fig. flUT a) correspond to small steps in all dimen- 
sions. Those in Fig. [TOT b) correspond to steps with one 
large component and D — 1 small components. Steps in 
Figs.nUtc) and (d) have, respectively, two and more than 
two large components. 
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FIG. 8: S r versus p for batch update, linear payoffs and bi- 
modal preference distribution (a) m = 1 and (b) m = 4. In 
both cases, N = 1001, s = 2, 1000 samples, and batch up- 
dates. 

As shown in Fig. QTJ attractors with small step sizes 
are dominant in the region of large p. Step sizes increase 
when p decreases, but the non-vanishing components do 
not increases isotropically. Rather, steps with one or two 
non-vanishing components become significant when p ~ 
0.2, and become more isotropic on further reduction of 
P- 

This sequential onset of isotropy is consistent with the 
cascade of dynamical transitions for the Gaussian on- 
line case when the diversity decreases. However, when m 
increases further, interferences between different signal 
dimensions will blur the anisotropic attractors. 



3. Attractor structure 

The structures of the attractors also change with p. 
For convenience, we take the example of m = 1, whose 
phase space can be plotted in two dimensions. From 
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FIG. 9: U\ and Vj versus p for batch update, linear payoffs 
and bimodal preference distribution, m = 1, s = 2, N = 1001, 
and 1000 samples. 

Fig. [TJJ we can see that the attractor structure with 
bimodal distribution is totally different from that with 
Gaussian distribution. For systems with Gaussian dis- 
tribution and batch update, the attractor has only two 
clusters of fixed points, between which systems oscillate 
(Fig. [HJa)), whereas for systems with bimodal distri- 
bution, the attractor visits many points located around 
an octagon in the small p phase (p < p c ), as shown in 
Fig- IT2T b). The system jumps among these points and 
occasionally stays at the origin. However in the large p 
phase (p > p c ) , the system will spend more time around 
the origin with occasional jumping out and back. 

4- Bursty dynamics 

Unlike the case of Gaussian distributions, the transi- 
tion from vanishing to non-vanishing step sizes on de- 
creasing diversity takes place in a bursty manner. For a 
given diversity, we define the activity as the fraction of 
time that the system stays away from the origin in the 
phase space. Fig. [T3] shows that the activity is low at 
high diversities, but increases to a high value when p is 
reduced below a critical value p c . This critical value p c 
depends on a. Estimating p as the point with an activity 
of 0.5 in Fig. lp c ~ 0.07 and 0.1 for a = 0.016 and 
0.032 respectively. We anticipate that p c approaches the 
value of 0.06 in the limit vanishing a, as proposed by Q. 

Looking deeper into the dynamics, we can find bursty 
behavior by introducing the payoff components k^it), 
which are related to the accumulative payoffs by 

n a (t) = J2^(tK- (io) 



up to time t. For m = 1, the payoff components ko and 
k\ also have different behaviors at low and high diversi- 
ties. As shown in Fig. RUTa) for low diversity, both ko 
and k\ oscillate around with large step sizes, resulting 
in the phase with non-vanishing variance. In contrast, 
Fig. HUTb) shows that for high diversities, the payoffs ac- 
cumulate with small step sizes, building up gradually to 
high values, and then return to low values with a huge 
step size. This bursty process resembles many natural 
phenomena in which energy is stored gradually and re- 
leased suddenly, such as earthquakes and volcano erup- 
tions. 



IV. MG WITH QUADRATIC PAYOFF 
FUNCTIONS 

To see how the behavior depends on the payoff func- 
tions, we change the payoff to the form of (p(x) = 
x 2 sgn(x) where x — \/~NA{t). Figure [TBI shows the gen- 
eral trend that a greater diversity gives a smaller variance 
of attendance and this effect is especially sensitive in the 
intermediate diversity region. It does not show any scal- 
ing behavior like in the step payoff model, nor any phase 
transition in the linear payoff model. Instead, it shows 
that a larger population always have a greater drop in 
the variance and a lower minimum variance. 

Besides, we are also interested to investigate how the 
initial position of the game affects its variance. First, 
1000 samples were simulated for each of the several dif- 
ferent diversities. Then the variance for the 1000 samples 
was arranged in ascending order and listed graphically in 
Fig.[H 

We can see that more and more samples have a rela- 
tively small variance as the diversity becomes larger. For 
each curve in Fig. I16[ the arrangement of the variance 
shows a gap in the variance, as pointed by an arrow. To 
facilitate the explanation, we define samples that are left 
to the gap to be " small variance" , while those that are 
right to the gap to be " large variance" . 

We find that these two groups of samples are strongly 
correlated with the initial states of the system. From 
Figs. [TTT a) to (d), it shows a general trend that the ini- 
tial positions of the small variance samples concentrate 
around the origin while those of the large variance sam- 
ples spread around. A basin boundary which is indicated 
by the lines appears among the light (orange) and dark 
(pink) dots and it becomes more recognizable at higher 
diversity. Also, we can notice that more samples with 
small variance and less samples with large variance ap- 
pear as the diversity increases. 

To determine the boundary analytically, we should 
look into the dynamics of the payoff function, Following 
the steps similar to those in Section III, we found 



In other words, /c M (t) is the total payoff of decision 1 of 
strategy a for signal p during the history of the game 



A"(i + 1) = A"(t) - J ^-(^/NA"(t)fsgnA^{t) (11) 
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FIG. 10: (a) to (d) Ranked step sizes for four classes of behavior with increasing isotropy, in batch update, linear payoffs and 
bimodal preference distribution, m = 3, s = 2, N = 1001, p = 0.04, and 200 samples. 



The square boundary can then be determined through 
the intersection between the Eq. (fTTjl and the lines y = 
—x. 



or 



(12) 



7rR 



NA^(t) 



(13) 



If the initial position of the system is located inside this 
basin boundary, that is, |iV^(0)| < V^rrR, A^{t) will 
eventually converge to the origin, which implies that the 
agents are not motivated to respond to the low payoffs. 
If the initial position of the system locates outside this 
basin boundary, A^{t) will eventually diverge, It implies 
that the agents are motivated to respond to high payoffs. 
Also, from Eq. (|13p . the boundary increases with diver- 
sity p. Thus, a larger diversity will give a smaller size of 
the basin of attraction for samples with large variance. 



After locating the basin boundary, we can compute the 
probability of finding attractors with small variance and 
large variance, respectively. Since the probability density 
function of the initial state y/NA^iQ) is Gaussian with 
mean = and variance = 1, the probability of finding 
small variance attractor is given by 



R 



small 



(erf0^) 2 



(14) 



and the probability of finding large variance attractor is 



Pi 



argc 



i-P s 



mall ■ 



This result is consistent with the simulation in that when 
the diversity is large, the probability of finding samples 
with small variance will be higher. A comparison between 
theory and simulation is given in Fig. 1181 which shows a 
good agreement. 
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p 



FIG. 11: The fraction of the four classes of step sizes as a 
function of p, in batch update, linear payoffs and bimodal 
preference distribution. Other parameters: same as those in 
Fig. [10] 



V. CONCLUSION 

We have studied the behavior of an adaptive popula- 
tion by using a payoff function that increases linearly and 
quadratically with the winning margin. We found in lin- 
ear payoffs, a continuous dynamical transition when the 
adaptation rate of the population was tuned by varying 
their diversity of preferences. This is in contrast with the 
case of payoff functions independent of the winning mar- 
gin, in which there is only a scaling relation between the 
variance and the diversity, and no phase transitions are 
found. The dynamical transition is due to the payoffs be- 
ing enhanced by large winning margins at low diversity. 
Furthermore, for systems with multi-dimensional signals 
feeding the strategies, we found a cascade of dynamical 
transitions in the responses to different signals, with the 
population variance increasing at each cascade. When 
the cascades are blurred at higher signal dimensions, a 
classification of the step vectors as done in Fig. QTj] also 
reveals the possibility of a crossover from anisotropic to 
isotropic motion in the phase space. 

We have also studied the effects of polarization of the 
initial preference distribution on the attractor behavior 
of the system by comparing the Gaussian and bimodal 
distributions. In the Gaussian case, there is a gradual in- 
crease in step sizes when the diversity decreases, whereas 
in the bimodal case, there is rather a sharp increase in 
the fraction of time with non- vanishing sizes in a narrow 
range of diversity, as shown in Fig. [T3J We also found 
that for bimodal distributions, the payoff components go 
through processes of accumulation and bursts. These 
observations illustrate that the more polarized the pref- 
erences are, the more erratic the dynamics is. There are 
many similarities shared among the different cases that 
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FIG. 12: The attractors of m = 1 in the phase space for batch 
update and linear payoffs: (a) Gaussian preference distribu- 
tion, p — 0.01; (b)bimodal distribution for small p (p = 0.004) 
and (c)large p (p = 0.04) respectively. Both for batch update. 



we have studied. For both Gaussian and bimodal dis- 
tributions, there are cascades for low signal dimensions 
which gradually disappear as signal dimensions increase. 
For online update, both endogenous and exogenous sig- 
nals result in similar macroscopic behaviors, for example, 
the diversity dependence of the variance of the buyer pop- 
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p 

FIG. 13: The activity as a function of diversity for bimodal 
distribution of preferences, batch update and linear payoffs. 

ulation and the ranked step sizes. 

For quadratic payoffs, a basin boundary separating two 
groups of samples is found. The group of vanishing step 
sizes adapts to the low payoff region of the quadratic 
payoff function and the group of non- vanishing step sizes 
adapts to the region with more rapidly increasing payoffs 
of the quadratic payoff function. 

In summary, we have found three pathways through 
which fluctuations increase with decreasing diversity. For 
linear payoffs with Gaussian preference distribution, fluc- 
tuations increase by increasing step sizes via cascades of 
continuous dynamical transitions. For linear payoffs with 
bimodal distributions, fluctuations increase is also due to 
the increase in the fraction of time with non-vanishing 
step sizes. For quadratic payoffs, fluctuations increase 
by enlarging the basin of the attractors with large fluc- 
tuations. This illustrates the rich behavior that the pop- 
ulation can self-organize in different environments. 

It is interesting to consider the phenomenology experi- 
enced by the agents at each side of the described dynam- 
ical transitions and crossovers. In this respect, we make 
the following observations. 

1) Agents using linear payoffs focus more of their atten- 
tion on opportunities of large winning margins, and tend 
more to neglect marginal wins. In markets with high 
diversity, agents with different preferences switch their 
strategies at different times, preventing the occurrence of 
a high volatility. Since the volatility is not large enough 
to induce strategy switches of such agents focusing more 
on large winning margins, the market is self-organized 
to a state of low volatility. However, in markets with 
low diversity, more agents switch their strategies at the 
same time, and the volatility is high. These profitable op- 
portunities are exploited by the agents more sensitive to 
large winning margins, and the market is self-organized 
to a state of high volatility. In markets with intermedi- 
ate diversity, the agents are selectively sensitive to some, 
but not all, of the signals in the market, leading to the 
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FIG. 14: The payoff components ko and fci as a function 
of time ( of a typical sample with bimodal distributions of 
preferences, batch update and linear payoffs. Parameters are 
m = 1, s = 2, and N = 1001, for (a) p = 0.01; (b) p = 0.4. 



cascades of phase transitions. 

2) For agents using quadratic payoffs, their interest on 
marginal wins vanishes even faster. Hence the market is 
self-organized to a state of low volatility when the initial 
volatility is low, irrespective of the diversity of prefer- 
ences of strategies. On the other hand, their emphasis on 
large winning margins rises even faster than linear pay- 
offs. Hence the market is self-organized to a state of high 
volatility when the initial volatility is high, irrespective 
of the diversity. Changes in the diversity cannot pre- 
clude the attractors of either high or low volatility. They 
merely influence the sizes of their basins of attraction. 

3) Both cases of linear and quadratic payoffs are dif- 
ferent from the case of step payoffs [20|, HlJ . Agents us- 
ing step payoffs place an equal emphasis on all winning 
opportunities irrespective of the winning margin. Con- 
sequently, the volatility does not vanish at any values 
of diversity, and there are no dynamical transitions or 
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FIG. 15: The dependence of variance on the diversity for 
online update, quadratic payoffs, endogenous dynamics and 
Gaussian preference distribution at (a) s = 2, m = 1 and 
averaged over 1000 samples, (b) s — 2, m = 2 and averaged 
over 1000 samples. 



multiple basins of attraction. Instead, a scaling relation 
between the volatility and the diversity is applicable. 

4) In markets with a bimodal distribution of initial 
preferences, the agents have polarized opinions about 
their responses to the signals. When the diversity is high, 
it takes a large number of time steps before the opinions 
of a group of agents are reversed. This gives rise to peri- 
ods of vanishing volatility, during which the system stays 
at the origin of the phase space. When the opinions of 
the agent group is eventually reversed, bursts of activities 
erupt. 

Recent attention was drawn to the role of the payoff 
function on reproducing realistic market behavior such as 
non-Gaussian features, the formation of sustained trends 
and bubbles, and intermittency [15, fl4|. Our study fur- 
ther confirms that tuning the payoff function and the 
preference distribution can lead to a rich spectrum of self- 



FIG. 16: The variance of 1000 samples arranged in ascend- 
ing order for p = 0.004, 0.008, 0.032, and 1, for online update, 
quadratic payoffs, endogenous dynamics and Gaussian pref- 
erence distribution. In all cases, N = 255, s = 2, and m — 1. 



organized states of the market. It would be interesting to 
consider populations of agents with different individual 
payoff functions and study how they interact. 



APPENDIX A: THE EQUATIONS OF MOTION 
IN MG WITH LINEAR PAYOFFS 

We consider the equations of motion in MG with lin- 
ear payoffs. Here, we focus on the case of a Gaussian 
distribution of initial preferences, and the generalization 
to the case of the bimodal distribution is straightforward. 
Using Eq. |T]), and averaging over Eq. ([3]), we obtain the 
step size for the historical state /x = 



a<b 

x[e(uj + n a (t 



did 



i)-n 6 (t + i)) 
e(w + n„(t)-n 6 (t))]. (Ai) 



Since the integral representation of the step function is 
given by 



dx 



0(2/) = 
Eq. (|A1|) becomes 



dp 
2T 



ip(x-y) 



(A2) 



a<b ' 



duj 



2/2R (e a - $) 



f-OC 

< / dx 
'o 



dp 

i P (n a (t)-n b (t)) 



ip(io-x) r e ip(Q (i+l)-n b (i+l)) 



(A3) 
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FIG. 17: (color online) The distribution of the variance related to its initial position for online update, quadratic payoffs, 
endogenous dynamics and Gaussian preference distribution, N = 255, s = 2, m = 1 and (a) p — 0.004. (b) p = 0.063. (c) 
p = 0.251. (d) p = 0.501. 



Decomposing the cumulative payoffs into payoff com- we arrive at 
ponents as defined in Eq. (|10[) . Eq. (|A3[) becomes 



(Ai4"(t)) = J™ dxj p2 ~ ipx x 

1 f duj _ w 2 /2/? JJ cos 2 pk v [ism2p(k,, - V^A^-i sin2pfc M ](A5) 



a<6 



r A [ * TT *P^(t)Ka-S?) Where ^ HaVe USed ideIltit y Co = °- 

/ 2tt 6 11 e Similarly, for the non-historical states v ^ we 

^ have 



Using the identity for = ±1 and x cos 2 p&\2z sinpfey cospA;^ 

3<0Kf-ef) = cos 2 q + i sin o cos _ ^) + sin 2 x [cos 2 p(fc M - ViVA^ 1 ) - cos 2 pfc^] . (A6) 
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FIG. 18: The comparison of the theoretical and simulative 
probabilities of finding samples with small variance for online 
update, quadratic payoffs, endogenous dynamics and Gaus- 
sian preference distribution, N = 255, s = 2, and m = 1. 



For m = 1, there are only two states. Let fx and 7* be 
the historical and non-historical states respectively. 
After evaluating the integrals, we obtain 



(AA"(t)) 



_ 1 
~ 8 

+erf 
—erf 



erf 



2fc M (t + 1) + 2fc F (<) 



2fc„ft + l) - 2fcg(t) 



V2i? 
2fc M (t) + 2%(t) 



2erf 



2A„(t + l) 



2i? 



/2i? 



-2erf 



2fc p (t) 
\/2R 



2fc_ M (t) - 2%(i) 



(A7) 



2i? 



<AA"(t)) 



erf 



2k ll (t + l) + 2k p (t) 
V2R 



2fc M (t + l) -2%(t) _ 2fc„(f) + 2%(t) 



+erf 



2i? 

2fc„(t) - 2%ft) 
V 7 ^ 



2i? 



(A8) 



where ^(i + 1) = k^t) - y/NA»{t). 
Now suppose /i = at t = 0. Following the dynamics 
sketched in Fig.[T!)l we can calculate fc^ and hp; as follows: 

fc o (l) = fco(0)-VAA°(0), fa(l) = fa(0) 
fc (2) = fc (l), k 1 (2) = k 1 (l)-VNA 1 (l) 
fc (3) = k (2), fci (3) = *i (2) - ViVA 1 (2) 
fc o (0) = fc (3) - VTVA (3), fci(0) = fcx(3) (A9) 
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-O t=2 
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FIG. 19: Attractor dynamics when (a) IAA 1 is larger and 
(b) | A A 1 is larger. 



Eqs. (|A7IIA9[) are the equations of motion in the phase 
space. When A 1 , A ~ N~ 1/2 and ki,k ~ N°, we 
obtain Eq. ^ for linear payoffs using Taylor expan- 
sion. On the other hand, there exist solutions with 
non- vanishing step sizes along one of the two directions, 
and vanishing step sizes along the remaining direction. 
Suppose AA° ~ TV -1 / 2 and ko(t) ~ h) = constant, 
and A A 1 ~ TV . From fci(0) = fei(3) in Eq. (|A"9"1) . we 
can get y/NA 1 ^) + VnA 1 (2) = 0. Since A 1 (2) = 
A 1 (l) + AA^l), we have 



and 



A\l) = — AA^l) 



A\2) = -AA\1) 



Let fci(0) = fci(l) = fei(3) = fa- y/NAA 1 (l)/4 and 
fa(2) = h + VNAA 1 (l)/4, where fa is the average value 
of fa{t). By Eq. JA7}, 



<AA X (1)) = - 



erf 



AA i (l)+4(A: 1 + fco)/V/V 



-erf 



A J 4 i (l)-4(fc 1 + fc )/V/V 



+2crf 



AA i (l)-4/ci/VA r 



Bp 



- erf 



+erf 



A J 4 i (l)-4(fc 1 -fc )/V/V 



8p 



+ 2erf A^(l)+4 fcl /VA 
A J 4 1 (l) + 4(fc^-fc )/VA 



(A10) 



In general, the dynamics converges with fa,ko ~ TV , 
yielding the self-consistent equation for A A 1 in Eq. ([5]) 
(we have made the dependence on t = 1 implicit). 

To find the variance, we use Eq. |5]). Knowing 
(A^ (t) ) = L4°(0) + + A x (2) + A°(3)]/4 = 0, the 

variance is calculated as: 

f! = ^[(-IaA^I)) 2 + (Iaa^i)) 2 ], 
TV 16 u 2 v " y 2 y " ' 

resulting in the expression for a 1 /N in Eq. (jSJ. 
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APPENDIX B: THE STABILITY OF THE STEP 
SIZE IN THE SECONDARY DIRECTION 

We consider the stability of the step size in the sec- 
ondary direction. Following the notation in Appendix A, 
we assume (AA 1 (1) ~ JV°) and /x = at t = 0, and the 
second direction is A . 

Suppose there is a small perturbation <5A°(0) at t = 0, 
then 

fc (l) = fc o (0) - VnSA°(0) = -y/N6A°(0). 



fcx(l) = MO) = h - ^W(l) « -^AA\l). 



We obtain 



1 , 



AA"(0) = -[-erf 



26A°(0) + ^AA 1 {1) 



Using Taylor expansion to the first order, 
AA°(0) = 

(1 + e *» )6A°(0) = 6A°(1). 



(Bl) 



A^O) = 0. 
Similarly for t = 1, 

fc (2) 



(B2) 



AT<L4°(0), &i(2) = ^A^(l) 



A^l 1 (l)=crf( — j+J-), AA°(1)=0. (B3) 



for t = 2, 

*o(3) 



7V,L4 (0), fci(3) 



A^(l) 



A.4 1 (2) = -erf( — LL2), Av4°(2) = 0. (B4) 



for t = 3, 

fco(4) = -2s/N5A°(0) - VnSA°(1) 



fcx(4) = 
AA°(3) 



AA^l) 

(1 + e —){SA°(0) + SA°(1)) 



AA X {$) = 0. (B5) 

(5^4° (4) is the accumulated perturbation from the previ- 
ous time steps. That is, 

A J 4°(4) = <L4°(0) + <L4°(1) 

1 (1 + e- {AAl £ )f )(5A°(a) + SA°(1)). (B6) 



V2^p 
leading to Eq. © 



APPENDIX C: THE EQUATIONS OF MOTION 
IN MG WITH BIMODAL PREFERENCE 
DISTRIBUTION 

For bimodal distribution of preferences of strategies, 
the calculation is similar to that of Appendix A, except 
changing P Ui from ' l 2R / 'V2^R to S(uj ia - VpiV)/2 + 
5(u ia + y/pN)/2. 

For m = 1, the theoretical result is as follows: De- 
noting the step functions by 0i = <d(x + \ff>N), O2 = 
Q(x — \fpN) : we have 

A„ = 0i(2fc M (t) + 2%(i)) + e 2 (2fc Al (t) + 2%(t)) 
-0i(-(2fc p (t) + 2%(i))) - e 2 (-(2fc M (t) + 2%(t))), 

B |t = 0i(2fc M (t))+e 2 (2fc p (t)) 
-0 1 (-2fc /1 (t))-0 2 (-2fc p (t)), 

<?„ - 0i(2* M (i) - 2%(t)) + e 2 (2fc„(t) - 2%(i)) 

-e 1 (-(2fc p (t) - 2%(i))) - e 2 (-(2^(t) - 2%(t))) 

(CI) 

For online update 

fc„(t + l) = *;„(*)- -v/WA^t), 
+ 1) = 



(C2) 



then 



AA»(t) = j^lMt + 1) + 2£ v(* + 1) + Ci.(* + !) 



-(A M (t) + 2fl^(t) + C M (t))]. 



1 



(C3) 



A^(i) = -L4 M (t+l)-C M (i + l) 
-(A M (t) - C M (t))]. (C4) 
Likewise, for batch update, with the notation ~p = 1—fi, 

A,, = Q 1 {2k ll {t) + 2%(i)) + 9 2 (2fc Al (t) + 2%(t)) 

-ei(-(2fc„(t) + 2%(t))) - e 2 (-(2fe M (t) + 2%(t))), 

S p = 9 1 (2fc M (<)) + 2 (2fc M (t)) 
-0 1 (-2^(t))-0 2 (-2fc M (t)), 

= e 1 (2fc M (t) - 2%(*)) + e 2 (2fc„(t) - 2%(t)) 
-e 1 (-(2fc p (t) - 2%(t))) - e 2 (-(2*; M (t) - 2%(t))) 

(C5) 

Since 



(C6) 



+ = *„(*)- VNA^t) 
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we obtain of Hong Kong (DAG05/06.SC36 and HKUST 603606). 

AA»(t) = jg[A M (i + 1) + 2B^(t + 1) + C„(t + 1) 
-(A^t)+2B^t) + C^t))}. (C7) 
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